The Ocean Aware project, led by Innovasea and funded through Canada’s Ocean Supercluster, is developing a fish passage observation platform to monitor fish without the use of traditional tags.This will provide an alternative to standard tracking technology, such as acoustic telemetry fish tracking, which are often not appropriate for tracking at-risk fish species protected by legislation.Rather, the observation platform uses a combination of sensors including acoustic devices, visual and active sonar, and optical cameras.This will enable more in-depth scientific research and better support regulatory monitoring of at-risk fish species in fish passages or marine energy sites.Analysis of this data will require a robust and accurate method to automatically detect fish, count fish, and classify them by species in real-time using both sonar and optical cameras.To meet this need, we developed and tested an automated real-time deep learning framework combining state of the art convolutional neural networks and Kalman filters.First, we showed that an adaptation of the widely used YOLO machine learning model can accurately detect and classify eight species of fish from a public high resolution DIDSON imaging sonar dataset captured from the Ocqueoc River in Michigan, USA.Although there has been extensive research in the literature identifying particular fish such as eel vs non-eel and seal vs fish, to our knowledge this is the first successful application of deep learning for classi...